Report of the MSE workshop, July 14-18, 2025

2025-07-27

Executive summary

The Jack Mackerel Management Strategy Evaluation (MSE) Technical Workshop brought together scientists, technical experts, and external reviewers to review recent progress and refine the MSE framework being developed under SPRFMO. The primary goal of the workshop was to ensure that the modeling framework and management procedures (MPs) are scientifically sound, technically transparent, and aligned with management priorities.

Key Outcomes and Advancements

  1. MSE Framework Consolidation
    Participants reviewed the jmMSE software package, confirming that it provides a robust and flexible platform for conducting MSEs. The package includes a reference set of operating models conditioned to historical data using MCMC, an efficient MP tuning algorithm, and tools for visualizing and comparing results.

  2. Robustness Testing
    The workshop clarified the role and scope of robustness tests. These tests are intended to explore how CMPs perform under a range of plausible yet uncertain scenarios rather than represent definitive alternative models. Scenarios reflecting changes in recruitment, spatial availability, environmental regime shifts (e.g., El Niño), and stock structure were reviewed and refined for implementation.

  3. Indicator-Driven MPs and HCR logic
    Empirical MPs based on one or more indicators were evaluated, with focus on two formulations:

    • TAC as a product of a target and a multiplier from an index.
    • TAC adjusted incrementally from the previous year based on index signals.

    We noted that the high current stock status (well within the “green” zone) tended to increase catch levels when tuning to achieve a desired P(Green). This can result in declining stock trends later in the projection period, even when the short-term performance criteria are met.

  4. Recommendations and Refinements
    The group recommended additional diagnostics and refinements, including:

    • Adding plots of how index trajectories relate to TACs.
    • Including new performance metrics that reflect stock status and trends in the final projection years.
    • Ensuring consistent treatment of selectivity, weights-at-age, and catch splits in both projections and reference point calculations.
    • Exploring robustness scenarios that account for variability in fleet selectivity and biological assumptions, particularly where CPUE is used as an input.
  5. Documentation and Transparency
    The group emphasized the importance of transparency in documenting model assumptions, data sources, and MP structure. The group agreed on priorities for improving documentation and sharing annotated examples of MP behavior.

  6. Next Steps and Implementation
    The next phase of work will focus on finalizing the candidate MPs, running the robustness tests, and summarizing trade-offs across key performance indicators. In discussions we also identified future reporting needs, including summary tables and figures for managers, and exploration of reference points and evaluation criteria beyond the green zone probability.

Introduction

Management Strategy Evaluation (MSE) has emerged as a critical tool for fisheries management, especially in contexts where data are limited or uncertainty is high. Foundational software frameworks like FLR were developed to facilitate reproducible, cross-disciplinary evaluation of management strategies through simulation and decision analysis (Laurence T. Kell and Scott 2007). Building on this foundation, recent advances have expanded FLR’s capacity for data-rich and data-limited systems alike, improving accessibility and integration with other tools (Hillary et al. 2023). Complementing these developments, a structured framework for evaluating methods and risk in data-limited fisheries has been proposed, providing practical guidance on the application of MSE in real-world settings (Carruthers et al. 2023).

The SCW15 Jack Mackerel MSE Technical Workshop was convened in response to the Scientific Committee’s request for progress on developing and evaluating management procedures (MPs) for jack mackerel under the SPRFMO framework. The meeting was held over five days (14–18 July 2025) and hosted in a hybrid format, with active participation from in-person attendees in Seattle and remote collaborators from SPRFMO Member States and invited experts. This event followed on previous technical work, including the SCW14 benchmark, and focused on finalizing the reference set of operating models (OMs), implementing robustness tests, and refining MP candidates using the jmMSE software package.

Throughout the week, participants engaged in live coding sessions, software validation, model tuning, and scenario refinement. The agenda was intentionally flexible, allowing the group to respond dynamically to technical challenges—such as issues with index generation, selectivity artifacts, and catch variability under different MP formulations. The workshop emphasized transparency, reproducibility, and documentation, with clear objectives to improve the utility and credibility of the MSE outputs ahead of Scientific Committee and Commission review.

By way of review, we provide a general outline for the workflow for defining and evaluating Management Procedures (MPs). We divided the process into three main stages (Figure 1).

Figure 1: Workflow for evaluating and selecting candidate management procedures (MPs).

The workshop participants recognized that all of the pieces for this exercise were available and implemented. However, the group struggled with getting the candidate MPs defined relative to available indices (“stage 1” in the diagram).

The appendices provide the participants participants, the agenda and the daily summary activities in the workshop minutes. The review from the external experts is also summarized in an appendix as well.

The following sections detail the discussion on how best to incorporate some environmental effects for projecting from the operating model. Specifically, we considered how to account for the effects of El Niño on recruitment and catchability/availability. We follow this with a sections reviewing the OM and then some results from applying existing and refined MPs. We conclude with a set of recommendations for the SC to consider.

Simulating El Niño effects in the Operating Model

To incorporate climate-driven variability into the Operating Model (OM) projections, we defined a scenario simulating El Niño–like events every five years beginning in 2030. These events affect primarily recruitment and distribution options. Based on the work of Iago and available literature, we proposed biological and fishery processes that may relate to El Niño conditions. The group discussed these and noted that they could be evaluated in the next round of stock assessment benchmark and for future MSE work.

The table below summarizes the proposed effects of simulated El Niño conditions on the OM, categorizing them by their expected direction, biological or fishery-based justification, and evaluation priority. The table is divided into two sections: effects that are prioritized for immediate evaluation and those deferred for further study.

The first items highlights two key El Niño-driven effects: A 30% increase in recruitment with a one-year lag, linked to ENSO-related early life stage survival (Figure 2). Shifts in catchability, with coastal regions experiencing increased availability and offshore regions seeing declines, reflecting observed onshore movement of fish during warm anomalies. This is also high-priority, with a focus on quantifying impacts on fishery removals.

The deferred effects were discussed and included the potential for reduced weight-at-age (potentially due to prey scarcity), earlier maturity (a stress response observed in small pelagics), and increased natural mortality (from predation or environmental stress). These are flagged for future study, pending historical data checks or further evidence. The table succinctly organizes hypotheses while clarifying immediate next steps for the OM framework.

Figure 2: Recruitment estimates and mean values (horizontal lines) used to estimate the impact of ENSO effects.

The following tables summarizes the key effects, their expected directions, and justification.

Effect Direction Justification
Recruitment ↑20% ↑ 1-year lag ENSO-linked early life stage effects on recruitment
Regional availability Coast catchability ↑ and offshore ↓ Onshore shift during warm anomalies

Discussed but deferred for further study

Effect Direction Justification Notes
Weight-at-age ↓Productivity Lower prey density and observed condition declines Check historical WAA anomalies
Age-1 maturity Earlier maturity Stress response seen in small pelagics similar impact on Recruitment
Coastal selectivity Age 1–2 sel ↑ Spatial contraction, availability change Confirm from CPUE by age?
M ↑30%/20% ↓ Survival Stress-induced mortality, predation

Estimating Relative Availability from Catch Proportions

Taking an assumption that over a recent period that changes in a smoothed proportion of catch by coastal and offshore areas roughly relates to the effective catchability \(q\) (which includes both true catchability and availability) of the fishery, i.e.:

\[ \text{Catch}_{\text{fleet}} \propto q_{\text{fleet}} \]

Thus, observed catch proportions can serve as a proxy for relative availability.

5-year moving averages of the proportion of the catch occurring in the “coastal” areas compared to the offshore fleet.
Year Range Coastal (%) Offshore (%)
2004–2008 81 19
2005–2009 78 22
2006–2010 74 26
2007–2011 76 24
2008–2012 78 22
2009–2013 82 18
2010–2014 84 16
2011–2015 86 14
2012–2016 86 14
2013–2017 85 15
2014–2018 86 14
2015–2019 87 13
2016–2020 91 9
2017–2021 93 7
2018–2022 94 6
2019–2023 94 6
2020–2024 94 6
Mean 85 15

Range of Change in Estimated Availability

Given this pattern we can assume a relative catchability due to an environmental effect. We note that the

  • Coastal effective catchability increased from a low of 74% (2006–2010) to a high of 94% (2018–2024), a +20 percentage point change.
  • Offshore effective catchability declined from 26% to 6%.

As a sensitivity, we could propose that the effective availability to the offshore fleet drops from 15% of the biomass (the mean) to gradually to 6% during El Niño periods (a 60% decline in \(q\) ). This would apply to the data generated for offshore CPUE index in the simulations. For the coastal zones, the effect of El Niño would correspond to an 11% increase in the availability of fish relative to the mean (85%). These changes would apply to the Chilean SC CPUE index and the Peruvian CPUE index data generation. This is one proposal among many that could be imagined. For example a slightly more conservative range could be based on the 10th and 90th percentiles of estimated effective catchability (from proportional catches):

Coastal:
•   10th percentile: 77%
•   90th percentile: 94%
Offshore:
•   10th percentile: 6%
•   90th percentile: 23%

These shifts may provide some scope for showing the impact of changes in the relative abundance indicators in index values. These reflect patterns over the past two decades, possibly due to environmental changes.

Review of the Operating Model specifications

The workshop reviewed the current specifications of the OMs. In particular, the assumptions for the reference point calculations were discusssed and contrasted with the 2024 assessment results and reports (South Pacific Regional Fisheries Management Organisation (2024)). Due to the terminal (2024) estimates of fisheries selectivities, the assessment report had anomalously high values for \(F_{MSY}\)

While the 2024 stock assessment produced high estimates of potential catch under the third tier of the harvest control rule—exceeding 4,900 kt based on \(F_{MSY}\)—this result was considered unrealistic due to likely upward bias in \(F_{{MSY}}\) estimates caused by strong selection on older fish. As a result, the Scientific Committee recommended constraining the 2025 TAC to be at or below 1,428 kt, representing only a 15% increase from 2024 levels and aligned with the Commission’s guidance. In developing the OM, reference points such as \(F_{{MSY}}\) were instead based on longer-term averages to avoid the influence of short-term variability or cohort effects, ensuring more stable and precautionary management advice consistent with the MSE framework (Figure 3).

Figure 3: Distribution of reference points from the operating model accepted by the workshop.

Several issues were identified with the current stock assessment that warrant further attention ahead of the next benchmark. Key among them are uncertainties in catch-at-age data stemming from differences in age determination methods across laboratories, as well as assumptions about mean body weight at age. The Scientific Committee emphasized the need for standardizing CPUE indices and improving data collection protocols, particularly regarding fleet-specific efficiency changes. Sensitivities to early age composition data—especially from the pre-1990 period—remain unresolved, with residual patterns noted for the North Chilean fleet. In addition, assumptions underlying selectivity and recruitment regimes were highlighted as critical sources of uncertainty, with substantial influence on reference points and management advice. Finally, the Committee underscored the importance of continued evaluation of single-stock versus two-stock model structures using simulation and MSE tools.

Summary of Workshop Outcomes

The SCW15 workshop provided a venue for progressing the Jack Mackerel MSE work, resolving some technical issues, and evaluating multiple MP configurations. A key outcome was the identification of problems in how MPs interacted with the OMs. This raised the need for either resolution of MP specification issues prior to the SC and/or holding a separate follow-up technical meeting, ideally in person. This would focus discussion on narrowing MP options. Depending on this direction, it may mean that such a meeting would have to occur after February 2026 and the Commission meeting.

Regarding the ability and facility for member scientists to use and evaluate the MSE framework, the group noted that the development of the jmMSE package was exceptionally well done. We found that difficulties inherent to the jack mackerel resource and assessment created unique problems. Specifically, the variable resource distribution, available data, and specifications of projection conditions (e.g., mean body mass-at-age, fishery selectivity at age) complicated how MPs could be evaluated. We noted that such specifications would be problematic for any other MSE framework as well.

Software and Technical Recommendations

  • Continue using FLR as the main MSE engine unless there is a dedicated effort to migrate to openMSE or another platform.

  • Improve naming conventions in code to reduce ambiguity. For example:

    • Functions like cpuescore2.ind and cpuescore3.ind could better reflect their purpose.

MSE Development Timeline and Deliverables

The group noted that MSE funding (in the form of providing support from external developers) may be available but would be contingent on:

  • Coordination with the current analyst (Iago).

  • Collaboration with the technical team.

  • Openness to using openMSE.

  • Clear timelines and deliverables.

Proposed deliverables and deadlines include:

  • Reference OMs (no multistock): End of July

  • Robustness OMs (no multistock): End of August

  • Shortcut calibration to the JJM assessment: End of August

  • Range of shortcut MPs run for all reference OMs: End of July

Planned products:

  • Technical documentation and reports:

    • Draft Technical Summary Document (TSD) by end of July.

    • Technical working papers and presentations for:

      • Shortcut calibration to JJM

      • Reference set OM results for MP archetypes

      • Robustness OM results

      • MP performance summaries

    • Slick MSE results summary.

    • TSD v1 by end of September.

Recommendations

For SC:

  • Adopt the current proposal structure with flexibility for future adjustment.

  • Recommend a shortlist of MP options to simplify the selection process at the Commission level.

  • Consider a placeholder method for calculating the 2026 TAC if MSE work is not yet finalized.

For Members:

  • Commit to a shared MSE software base (FLR or openMSE).

  • Engage in pre-SC online meetings to broaden participation in MSE discussions.

For Analyst (Iago):

  • Prioritize enhancements discussed during the workshop:

    • Code clarity and naming conventions

    • Logical parameter usage across MPs

    • Refinement of FLR-to-dataframe functions

  • Identify successor strategy after contract ends in 2025.

Jack mackerel MSE workplan

During the 2025 Commission meeting (COMM13), Chile proposed a deviation from the established Management Procedure (MP) guidance by recommending a Total Allowable Catch (TAC) for Trachurus murphyi of 1.785 million tonnes for 20251. This proposal exceeded the 15% year-on-year TAC change limit adopted by the Commission as part of its management framework2. Chile based its proposal on Article 4.3 of the Convention, asserting the need to consider measures by coastal states and highlighting that the stock had remained above MSY reference levels for over five consecutive years3.

Chile also cited Scientific Committee (SC) analysis, particularly paragraph 136 of the SC12 Report, which indicated that the stock was in Tier 3 of the adopted Harvest Control Rule (HCR), theoretically allowing for a much higher TAC (up to 4.997 Mt) if the full HCR were applied4. However, as per the Commission’s directive to the SC (COMM3-Annex C), catch advice was constrained to not exceed a 15% increase, consistent with the MP in place5.

The Scientific Committee Chair emphasized in plenary that the SC’s advice adhered to the current MP, limiting TAC changes to ±15% to account for uncertainties, including potential model misspecification. Paragraphs 136 and 137 of the SC12 report note that this constraint was not intended to account for rebuilding potential, but rather to ensure precautionary stability and reduce the risk of large interannual TAC fluctuations6.

Chile revised its proposal multiple times (COMM13-Prop01_rev1 through rev5) to address Member concerns, including scientific justification, process adherence, and future management stability. Nevertheless, many Members expressed concern that implementing a TAC increase beyond 15% ahead of finalizing the MSE would undermine the credibility of the HCR framework and create challenges for future rule-based decisions.

Chile also referred to Table 37 of Annex 7 of the SC12 report, which showed that biomass remained above target levels under various fishing scenarios. However, the SC Chair cautioned that these projections were illustrative only and not intended for management advice7. The SC further stressed that alternative scenarios outside the adopted HCR should be evaluated through MSE before informing decisions.

Following intensive discussions and compromises, the Commission adopted a revised proposal (COMM13-Prop01_rev5), establishing a one-year TAC of 1.785 Mt for 2025 while reaffirming:

  • The importance of completing the MSE and adopting a revised MP and HCR by 2026;
  • The temporary nature of the 2025 deviation;
  • The application of Article 20(5) of the Convention to enable responsive management should the stock decline8.

This episode highlighted both the limitations of static MP rules under changing stock and socioeconomic conditions, and the importance of timely completion of the MSE process to provide more flexible and scientifically robust guidance.

Footnotes

Near term

Participants were encouraged to document their activities during the workshop, including the methods explored and tuning targets used. Work tasked identified included:

  • All continue to evaluate MPs to the extent practical ensuring that they can be tuned to achieve a 60% green status and are consistent with the available OM data stream projections.

  • Jim evaluated 9 MPs (including bufferdelta2, cpuescore2, test acoustic, and combinations of CPUE indices with different delta_TAC values), all tuned to achieve 60% green status.

  • Jose/Chile apply shortcut tuning methods to reach similar green zone targets.

This table illustrates how an array of MPs might be evaluated for summarization:

Year Method Metric Tuning Parameter Other Parameters Score Index Comments
2024+ buffer.hcr depletion target bufflow, buffup, limit cpuescore3.ind Original pkg function
2024+ bufferdelta.hcr depletion width sloperatio cpuescore3.ind Modified; not compatible with z-score metrics
2024+ bufferdelta2.hcr zscore width sloperatio cpuescore2.ind New; not compatible with depletion
2024+ buffer2.hcr zscore target width (affects buffer) cpuescore2.ind Original; adjusted for zscore (limit = -2 SD)

Medium term

The group agreed to continue working on the MSE framework, with a focus on the following tasks: - Consider how reference set of OMs could be regenerated conditioned to historical data using MCMC methods. - Implement robustness tests to evaluate how MPs perform under a range of plausible yet uncertain scenarios. - Refine MP candidates using the jmMSE software package, ensuring they are scientifically sound and technically transparent. - Document and share the MSE framework, including model assumptions, data sources, and MP structure. - Explore additional diagnostics and refinements, including new performance metrics that reflect stock status and trends in the final projection years.

Conclusion

References

Carruthers, Thomas R., Quang C. Huynh, Adrian R. Hordyk, David Newman, Anthony D. M. Smith, Keith J. Sainsbury, Kevin Stokes, et al. 2023. “Method Evaluation and Risk Assessment: A Framework for Evaluating Management Strategies for Data-Limited Fisheries.” Fish and Fisheries 24 (6): 1335–50. https://doi.org/10.1111/faf.12726.
Hillary, Richard M., José M. Castro, James T. Thorson, Sean C. Anderson, and Laurence T. Kell. 2023. “The FLR Software Framework for Building Management Strategy Evaluation Systems: Recent Advances and Application to Data-Rich and Data-Limited Fisheries.” Fisheries Research 263: 106585. https://doi.org/10.1016/j.fishres.2023.106585.
Laurence T. Kell, Paul Grosjean, Iago Mosqueira, and R. D. Scott. 2007. “FLR: An Open-Source Framework for the Evaluation and Development of Management Strategies.” ICES Journal of Marine Science 64 (4): 640–46. https://doi.org/10.1093/icesjms/fsm012.
South Pacific Regional Fisheries Management Organisation. 2024. SC12 Report Annex 07: Jack Mackerel Technical Annex.” SPRFMO; https://sprfmo.int/assets/Meetings/02-SC/12th-SC-2024/SC12-Report-Annex-07-JM-Technical-Annex.pdf.

Footnotes

  1. COMM13-Prop01 and COMM13-Prop01_rev5 (Annex 10d)↩︎

  2. COMM3-Annex C↩︎

  3. SPRFMO Convention, Article 4.3↩︎

  4. SC12 Report, Paragraph 136↩︎

  5. COMM3-Annex C↩︎

  6. SC12 Report, Paragraph 137↩︎

  7. SC12 Report, Table 37 of Annex 7↩︎

  8. SPRFMO Convention, Article 20(5)↩︎